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Abstract 

Several passive microwave satellites orbit the Earth and 
measure rainfall. These measurements have the advantage 
of almost full global coverage when compared to surface 
rain gauges. However, these satellites have low temporal 
revisit and missing data over some regions. Image fusion 
is a useful technique to fill in the gaps of one image (one 
satellite measurement) using another one. The proposed al- 
gorithm uses an iterative fusion scheme to integrate infor- 
mation from two satellite measurements. The algorithm is 
implemented on two datasets for 7 years of half -hourly data. 
The results show significant improvements in rain detection 
and rain intensity in the merged measurements. 



1. Introduction 

Satellite monitoring of rainfall has the advantage of al- 
most global areal coverage compared with ground measure- 
ments (gauge and radar). Satellite rainfall retrievals have 
enormous potential benefits as input to hydrological and 
agricultural models because of their real time availability, 
low cost, and full spatial coverage IH. One category of 
these measurements are carried out by Passive Microwave 
(PMW) sensors in Low Earth Orbit (LEO) platforms. PMW 
measurements of rainfall have their own unique advantages 
as well as shortcomings. These sensors have a low tempo- 
ral revisit (usually two times a day) because of their orbital 
characteristics. Moreover, in some cases they have miss- 
ing data over some regions where retrieval is confounded 
by other influences. Therefore, in order to obtain reason- 
ably accurate rainfall estimation and with a good spatial and 
temporal sampling resolution, useful for studies of climate 
change and large-scale hydrological processes, it is neces- 
sary to merge different types of measurements. 

It is not possible to fill in the gaps of one measurement 
(hereafter, called image) using another one because none of 
these images are perfect measurements, and this will cause 
discontinuities in the final image. In addition, the compari- 



son of results with the results of an interpolation algorithm 
(which produces a merged image by interpolating between 
two input images) shows that a simple interpolation is not 
enough to merge the two images, and the results are not ac- 
curate. 

In order to produce a better estimation of the rainfall, 
researchers have tried to combine different types of rain- 
fall measurement. Several models have been developed that 
combines satellite measurements with ground-base mea- 
surements, and provide estimation of rainfall for both miss- 
ing pixels and for the times that there has not been a satel- 
lite measurement. Some of these models include: Global 
Precipitation Climatology Project (GPCP) |7|, Precipitation 
Estimation from Remotely Sensed Information using Arti- 
ficial Neural Networks (PERSIANN) |5|, Climate predic- 
tion center MORPHing method (CMORPH) |6|, and CH- 
mate prediction center Merged Analysis of Precipitation 
(CMAP) 1 1 1 1. Shen et al. (2010) investigated the errors and 
biases of these merged measurements 1 10] and showed that 
they are not reliable in some regions. Here, I will only fo- 
cus on improving the retrievals when there are at least two 
satellite retrievals available over one region. However, this 
method can be applied to any two sets of rainfall measure- 
ment to fill in the missing regions. 

One of the methods that can be used for image fusing 
is multi-resolution pyramids method |T1. This method is 
based on fusing the images in a transformed domain, cre- 
ated by decomposing images into sub-images. In this pa- 
per, I propose an algorithm for merging two rainfall im- 
ages using the multi-resolution pyramid method and an in- 
terpolation scheme. The proposed algorithm has two sec- 
tion: Texture Production and Shape Production. In order 
to produce the texture, it uses the multi-resolution pyramid 
method, and for producing the shape, it uses an interpola- 
tion scheme. The results show that the distinction between 
shape and texture in the algorithm improves the results both 
in detecting the rainfall and in producing the accurate rain 
intensity. 

The paper has four section. First, the data sets that were 
used are described. Then, the algorithm is presented in de- 
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Figure 1 . Procedure of the proposed algorithm. Rain Shape is an 
image of [-1, 0, 1]. Other images show rain intensity (from blue 
to red the intensity increases from zero to its maximum; dark blue 
are the missing regions) 



tail. Next, results are provided with comparisons with other 
methods. Last, a discussion on the results is presented. 

1.1. Data Sets 

The satellite measurements of rainfall for this study 
are obtained from National Oceanic and Atmospheric 
Administration (NOAA) satellites 15 and 16 (NOAA-15 
and NOAA- 16) and Tropical Rainfall Measuring Mission 
(TRMM). The former is produced by Microwave Surface 
and Precipitation Products System (MSPPS) in NOAA and 
the latter is produced by Goddard Earth Sciences Data and 
Information Services Center in National Aeronautics and 
Space Administration (NASA). The rainfall rates are from 
the AMSU-B sensors on board the NOAA-15 and NOAA- 
16 and the TRMM Microwave Imager (TMI) on board the 
TRMM satellite. An independent set of surface-based radar 
measurements produced by NEXt-Generation RADar Net- 
work (NEXRAD-IV) is assumed as the true measurement 
and used to evaluate the results of the model. This assump- 
tion can be further investigated by incorporating the errors 
of the radar measurements. 

For this study, all the data are mapped to a common 
spatial gird of 0.2b°latitude x 0.25° longitude covering 
the Central U.S. region ranging 26.10° - 42. 10° AT and 
107.85° - 91.85°iy. The period of study was from Jan. 
2004 until Dec. 2010. All the data are half-hourly measure- 
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Figure 2. The pyramids algorithm for producing texture (the Fu- 
sion Algorithm is described in Figured 



ments. 

In order to obtain a good dataset, I selected images that 
had a reasonable number (at least 40) of non-missing pix- 
els. Imposing this criteria, a set of 219 coincident images 
for each of the NOAA- 16 and TRMM, and a set of 221 co- 
incident images for each of the NOAA-15 and TRMM was 
created. It should be noted that each sample is a 64 x 64 
pixel image. 

2. Methodology 

The procedure for producing the merged images consists 
of two steps: texture simulation and shape simulation. Here, 
texture is the rain rate (or intensity) and shape is the rain 
support (i.e. rain / no rain areas.) The complete procedure is 
illustrated in Figure[T] Using the two input images, a texture 
and a shape is produced using the methods described in the 
following. Then, the fused image is created by multiplying 
the texture and the shape. 

2.1. Texture Production 

Image fusion using pyramids transformation is a use- 
ful method to fill in the gaps of one image using another 
one(s). This method that was first proposed by Adelson et 
al. (1984) uses an iterative fusion scheme to integrate in- 
formation from multiple images 1 1 1. Figure |2] shows the al- 
gorithm for this method. First, a filter is applied to the two 
input images to decompose them into sub-images (or equiv- 
alently to coefficients). Then, a fusion algorithm is applied 
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Figure 3. Structure of the Steerable pyramids in frequency domain 
(soruce: Liu et al. 1 8 1) 
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to merge the subimages and produce a fused subimage. Fi- 
nally, an inverse transformation will produce the merged 
image. 

Rainfall is an intermittent event in space, so choose of 
the filter is one of the important aspects in this work. It is 
necessary to have a decomposition that can obtain the most 
information possible from the images. Preliminary exper- 
iments using Gaussian, Laplacian, and Steerable pyramids 
showed that Steerable pyramids works better in decompos- 
ing the images, and the resulting fused images are more ac- 
curate. Thus, I will only use the Steerable pyramids in the 
following. 

Steerable pyramids is a linear multi-scale, multi- 
orientation and self-inverting image decomposition intro- 
duced by Freeman and Adelson |3|. Using Steerable filters 
Lq and i^o, the image is first decomposed into low-pass and 
high-pass subbands. The low-pass band continues to be di- 
vided into a set of oriented bandpass subbands Bq,- - • , Bn 
and a lower low-pass subband Li. The lower low-pass sub- 
band is sub-sampled by a factor of 2 along the x and y di- 
rections. Repeating this procedure, further decomposition is 
achieved. Figure [3] shows the structure of this pyramid. The 
steerable pyramid representation is translation-invariant and 
rotation-invariant. 

Steerable pyramids has been widely used in vision prob- 
lems and image analysis. Here, I use it to transform the 
initial images to a sub-image domain and inverse-transform 
the fused sub-images to the merged final image. Testing dif- 
ferent levels and orientation of decomposition, I found that 
a decomposition level of 4 (which is the maximum possi- 
ble) and a decomposition orientation of 16 is good enough 
to decompose the images. 

As it was shown in Figure[2j a fusion algorithm is needed 
to combine the two sub-images (or coefficient sets) in the 
transformed domain. A simple criteria for this integration 
is absolute value maximum selection (AVMS). This crite- 
ria selects the larger absolute values in the sub-images as 
the corresponding value for the fused sub-image. Although 
this criteria works well in many applications, it is so simple 
and it might miss some of the details in the integration pro- 
cess. Liu et al. (2001) proposed a fusion algorithm using 



Figure 4. Fusion algorithm in the decomposed domain 



the Laplacian pyramids |8|. In this algorithm each pair of 
subimages from the two input images are decomposed by 
using the Laplacian filters. Then, an integration scheme is 
applied to merge the two pyramids of the subimages. This 
integration scheme can be AVMS or any other defined crite- 
ria. Next, the fused subimage is produced by inverse trans- 
formation of the integrated pyramids of the input subim- 
ages. This fusion algorithm allows for more information 
to be fused into the merged subimages, and for the rainfall 
images that are so intermittent is really necessary. There- 
fore, I use this fusion algorithm for merging the subimages. 
Figure [4] shows the structure of the fusion algorithm. 

2.2. Shape Production 

This part of the algorithm will produce shapes for the 
rainfall fileds (i.e. support). Early investigations on using 
the pyramids method to merge the two rainfall images re- 
vealed that the method will produce a kind of continuous 
texture in the resulting image, which is not the case in real- 
ity. Moreover, the pyramids method produces high values of 
false alarm in the fused images. Therefore, I implemented 
this part to constrain the rainy areas in specific regions of 
the final image. Nevertheless, these regions are still defined 
by the two input images, but using an interpolation scheme. 

The procedure is that a linear interpolation is applied to 
create a merged image of the two input ones. Then, the 
pixels that were missing in both of the input images are set 
to be missing in the final image too. This is because of the 
high intermittency of rainfall patterns that makes it hard to 
interpolate the value of the missing pixels from the non- 
missing ones. 

This part produces an image of values of [-1, 0, 1] in 
which -1 means missing pixel, means no rain can occur, 
and 1 means rain can occur. Conditioning the texture to be 
only inside the possible rain areas, the fused measurement 
is produced. 
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Figure 5. CDF plot of detection statistics for NOAA-16 and 
TRMM dataset 



2.3. Fused Image Production 

This step is a multiplication of the produced texture and 
the produced shape. Therefore, the regions that have a shape 
value equal to 1 will have their values from the texture prod- 
uct, the regions that have a shape value equal to zero will be 
zero in the fused image, and the regions that their shape 
value is -1 (missing data) will remain missing in the fused 
image. 

3. Results 

In this section the results of the proposed algorithm are 
presented and evaluated for both rain detection and rain in- 
tensity. Ebert (2007) provides a comprehensive review on 
the statistics useful for validating the results of a rainfall 
estimation methods |2|. Here, I selected three statistics to 
evaluate the capability of the algorithm in detecting rainfall. 
The statistics are: Probability of Detection (POD), False 
Alarm Ratio (FAR), and Threat Score (TS). They are de- 
fined as following: 
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Figure 6. PDF and CDF of rain intensity for NOAA-16 and 
TRMM dataset 
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hits -\- misses -\- false alarms 

in which hits is defined as the number of instances that 
both of the true measurement and the proposed measure- 
ment detect rain; misses is the number of instances that the 
true detects rain, but the proposed one does not; and false 
alarms is defined as the number of instances that the true 
does not detect rain, but the proposed one does. Note that 
I have only considered the detection of rain. Equivalently, 
it is possible to check the score for detection of no-rain, but 
as I consider both POD and FAR of rain it is not necessary 
to evaluate the no-rain detection score. 

The preferred values for each of these statistics are POD 
= 1, FAR = 0, and TS = 1. Figure [5] shows the Cumulative 
Distribution Function (CDF) of the above mentioned statis- 
tics for NOAA-16 and TRMM dataset (with the characteris- 
tics described in section pTT] ) using the developed algorithm 
in this paper, the single interpolation method, and the pyra- 
mids method. As this figure shows, the pyramids method 
works well in improving the POD and TS, but at the same 
time it worsens the FAR to a high degree. Therefore, it is 
detecting the rainy areas by overestimating the rainy regions 
and giving too much false alarm that is not acceptable. The 
interpolation method seems to have a well matched result 
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Figure 7. CDF plot of detection statistics for NOAA-15 and 
TRMM dataset 



with the fused algorithm. They both improve the POD and 
TS with respect to the two input measurements (NOAA- 
16 and TRMM). Moreover, they provide a reasonable FAR 
which is somehow similar to the two input images. How- 
ever, the difference between the interpolation method and 
the fused method becomes clear by comparing the Probabil- 
ity Distribution Function (PDF) and CDF of the rain inten- 
sity of different methods. This is an important comparison 
because a method that is only capable of detecting rainfall 
and does not provide reasonable intensities will not be use- 
ful. Figure |6] shows the PDF and CDF of rain intensity for 
NOAA-16 and TRMM dataset. Note, these plots are for the 
pixels that were non-missing in all the measurements {i.e. 
True, NOAA-16, TRMM, Fused, Interpolation, Pyramids.) 
As can be seen in Figure |6] the PDF and CDF of the 
fused measurement match well with the ones for true mea- 
surement, which is the surface-based radar measurement as 
discussed in section [LT] In addition, to test which of these 
CDFs are from the same distribution as the true one, I run 
the Kolmogorov-Smirnov test O. This test evaluates the 
hypothesis that two given datasets are from the same con- 
tinuous distribution. The result with 5% significance level 
showed that only the fused measurement can be from the 
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Figure 8. PDF and CDF of rain intensity for NOAA-15 and 
TRMM dataset 



same distribution as the true one. This shows that use of the 
fused algorithm, proposed in this paper, improves each of 
the pyramids and interpolation methods and provides bet- 
ter estimation with respect to both rain detection and rain 
intensity. 

In order to further test the method, I run the method 
on another dataset which was the NOAA-15 and TRMM 
dataset (with the characteristics described in section |1.1| ) 
The results are illustrated in Figures [7] and [8] As can be 
seen in these two images, the proposed fusion algorithm 
produces good results in this case too. The Kolmogorov- 
Smirnov test again showed that only the fused measurement 
CDF can be from the same distribution as the true one, at 
5% significance level. This proves the capability of the pro- 
posed algorithm in producing better estimations of rainfall 
for other satellites too. 

Figures [9] and [T0| show four samples of the sources, 
fused, and true images of rainfall intensity in NOAA-16 and 
NOAA-15 dataset. The purple color in these images repre- 
sents the missing regions in the images. 



4. Discussion 

This paper explored the possibility of using a multi- 
scale imagery technique in accordance with an interpola- 
tion scheme to merge two satellite measurements of rainfall 
and produce a better estimation of rain intensity with less 
missing points in the fused image. The proposed algorithm 
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Figure 9. Sample source, fused, and true measurements of NOAA- 
16 and TRMM dataset (the scale shows rain intensity in mm/hr) 
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Figure 10. Sample source, fused, and true measurements of 
NOAA-1 5 and TRMM dataset (the scale shows rain intensity in 
mm/hr) 



improves both rain intensity and rain detection of the two 
input images. Moreover, it was shown that only the results 
of the rain intensity from the fused image can be considered 
to be from the same distribution as the true measurement. 



In addition, as can be seen in Figure [T0| there are re- 
gions that there is no true measurement, and this algorithm 
can improve the satellite measurements and provide accu- 
rate estimates over them. Indeed, other the 50 sates of the 
US which have full surface-based radar coverage there are 
very few regions in the world that have a similar full cov- 
erage. Therefore, this method can be used in a any place 
over the globe to fuse different satellite measurements and 
produce better estimations of rainfall. 

Furthermore, daily estimates of satellite rainfall can be 
improved by using this algorithm. There are places on the 
world that are not covered in the daily passes of one satel- 
lite, but they are covered by another satellites which miss 
some regions in the first one. The proposed algorithm can 
be applied to these daily measurements to provide full daily 
rainfall estimates that are useful for water balance studies 



and rainfall forecasting. 
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